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ABSTRACT 



The purpose of this paper is to develop a 
definitional framework for the constrained 
optimization of an objective function with stochastic 
parameters. Included in this definitional framework 
are the construction of a deterministic reformulation 
for the stochastic problem, guidelines for selecting 
an algorithm for optimizing the deterministic 
reformulation and a criterion, called invariance, 
which is used for solution sensitivity analysis in the 
context of stochastic parameter behavior. 
Illustration of the application of the definitional 
framework and an analysis of the conditions imposed by 
the invariance criterion is provided for linear, 
piecewise linear, and quadratic functions. 
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I. 



I NTRODOCTI ON 



The overhaul of a naval vessel is a three phase event; a 
planning phase of approximately one year, an industrial work 
phase, and a post-overhaul and inspection phase. The 
typical overhaul has a budget in the millions of dollars and 
involves several thousand individual jobs which are normally 
aggregated into a much smaller number of managerial groups. 
In this paper these managerial groups will be referred to as 
stages. 

A major event in the overhaul planning phase is the job 
screening conference. The purpose of this conference is to 
identify the stages in which advance planning and 
procurement should commence. This is accomplished by 
individually reviewing the proposed jobs in each of the 
stages and selecting, within stage and overall budget 
limits, those jobs which will be accomplished during the 
overhaul. This screening process is, in effect, an implicit 
approach toward an optimal allocation of the available 
resources. As these resources are increasingly limited 
there is growing interest in formalizing the screening 
process into an explicit optimization which uses the 
standard tools of mathematical programming. One formidable 
obstacle in the development of the explicit optimization 
model is the identification of a measure of effectiveness 
which is applicable to each of the stages. Even assuming 
the existence of some measure there remains a basic problem 
which arises from the method of determining return as a 
function of resource allocation for each stage. These stage 
return functions will rarely be explicitly known and 
therefore must be estimated from available data. Regardless 
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of the quantity and quality of the data the estimation will 
result in stage return functions with parameters subject to 
random error. Thus, in addition to identification of a 
measure of effectiveness, a prerequisite of explicit 
optimization of overhaul planning allocations is an ability 
to conduct constrained optimization of an objective function 
whose parameters are subject to random error. 

The optimization of an objective function with 
parameters subject to random error falls into the general 
realm of stochastic programming. As noted by Nemhauser [8], 
the concept of optimization in the presence of stochastic 
behavior is not well defined, and is the subject of 
continuing discussion. From this discussion two definitional 
frameworks have arisen. One is highly theoretical and is 
exemplified by statistical decision and game theory. The 
other framework is more computationally oriented and is 
exemplified by the method which addresses the creation of a 
deterministic reformulation of the stochastic program to 
which the optimization can be applied. There are several 
variants but the central idea of a deterministic 
reformulation is to select a particular value from the range 
of each of the stochastic parameters, a common choice is the 
expectation, and to treat these values as deterministic 
parameters for the purpose of optimization. The approach is 
open to considerable criticism, the most common being the 
failure to account for possible effects on the solution 
caused by differences between the realizations of the random 
variables and the values used in the optimization. 

The goal of this research is to develop, for an 
arbitrary measure of ef f ect iveness, a definitional framework 
for the optimization of the general class of stochastic 
programs posed by the overhaul planning problem. The 
framework is to be computationally oriented and based on a 
deterministic reformulation. The method defines a concept 
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called ’’invariance" which is used to assess the effect of 
stochastic parameter behavior on the set of stages in which 
no advance planning and procurement actions are indicated. 
The criterion is directed toward this set rather than the 
set of stages in which advance planning and procurement 
should commence since the costs associated with the 
erroneous inclusion of a stage in the 
no-planning-and-procurement set are much larger than the 
costs associated with the erroneous inclusion of a stage in 
the advance- planning-and-procurement set. 

Chapter two will develop a mathematical model of the 
overhaul planning problem, define the invariance criterion 
and develcp the conditions imposed by this criterion on the 
deterministic reformulation and its optimization. Based on 
these conditions a solution approach for the optimization of 
the overhaul planning problem is proposed and its 
application illustrated in chapters three, four, and five 
for linear, piecewise linear, and quadratic stage return 
functions. 



/ 
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II. PROBLEM FORMULATION AND CONCEPT DEVELOPMENT 



A. MATHEMATICAL FORMULATION 

Assuming that there are n stages to be considered by the 
screening conference a mathematical formulation of the 
overhaul planning problem is dependent on the identification 
of a composition operator used to form the overall objective 
function from the individual stage return functions. As the 
overhaul planning problem is of an economic nature, that is: 
it addresses the allocation of scarce resources, any 
arbitrary measure of effectiveness is, in effect, a form of 
utility measure. A reasonable choice for the composition of 
the stage returns into an aggregate measure of the utility 
of the total overhaul package is summation. That is, stage 
returns are assumed additive. Based on these assumptions 
one mathematical formulation of the overhaul planning 
problem is: 



n 



♦ 

MAX r (d ) 

j=1 j j 




n 



S/t H 



^ d < BD 

j=1 j 



0 < d < ub j=T,2 

j j 



where 



th 



r (d ) is the j 

j j 



stage stochastic return function 
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th 

d is the j stage decision variable. 

j 

BD is the total budget constraint, 
th 

ub is the j stage budget constraint. 

j 

Based on the economic context of the problem, it is assumed 
that stage allocations are subject to diminishing returns, 
thus the stage return functions are concave. In the same 
context it is also reasonable to assert that the sum of the 
individual stage budget constraints exceeds the total budget 
constraint, thus at optimality the total budget constraint 
will be binding. 



B. DEFINITIONAL FRAMEWOBK 



• Deterministi c ^age Returns 



As previously noted the concept of optimization in 
the presence of stochastic behavior is not well defined. The 
stochastic optimization problem formulated above is to be 

defined in terms of a deterministic reformulation, each 

* / 

r (d ) being converted to. a deterministic r (d ) by 

j j j j 

selecting from the respective ranges a specific value for 

each of the stochastic parameters. The optimization of the 
deterministic reformulation is to be achieved by application 
of one of the existing algorithms of mathematical 
programming. The specific algorithm will depend on the form 
of the stage return functions used in the deterministic 
reformulation. 
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2 . 



Solu^on S ensitivit y An a lysis 



The effect of stochastic parameter behavior on the 
optimal solution of the deterministic reformulation can be 
observed from three viewpoints, in terms of the values of 
the decision variables and the value of the objective 
function, in terms of the solution set or in terms of the 
complement of the solution set. 



The solution set S for an n-variable 

m-constraint problem is defined as the 
set of variables different from zero. 

The complement of the solution set, 
0 

denoted S , is the set of variables 
which are zero. 



As this problem is in a planning context the solution 

•f 

set S , which identifies the stages in which advance 

planning and procurement should commence, and its complement 
0 

S , which identifies the stages in which there should be no 

advance planning and procurement, are more important than 
the values of the decision variables or the value of the 
objective function. Since the costs associated with a stage 

0 

being incorrectly included in S are much larger than the 
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the 



+ 

cost of a stage being incorrectly included in S , 

sensitivity analysis of the solution to the deterministic 
reformulation should be directed toward the complement of 
the solution set. Invariance, defined below, is the 
criteria which will be used to assess the sensitivity of the 
complement of the solution set to the stochastic parameter 
behavior . 

The optimal solution of the 
deterministic reformulation is invariant 
with respect to the specified ranges of 
the stochastic parameters, if within 
those specified ranges no zero valued 
variable is required to depart the 
ccuplement of the solution set, that is 
become positive, to maintain optimality. 

Basically for a solution set to be invariant the variables 
of the deterministic reformulation which have an optimal 
value of zero must remain at zero for parametric 
perturbations throughout the specified ranges of the 
stochastic parameters. There are two concepts in the 
definition of invariance which require additional 
development. 



a. Specified Parameter Range 



Invariance is defined in terms of specified 
ranges for each of the stochastic parameters. Some of these 
parameters may be the result of a statistical regression and 
as such they are assumed to be normally distributed random 
variables with a nominal range of (-00,00) . Ranges of this 
magnitude would always violate the invariance criterion. 
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thus one requirement for meaningful analysis of solution 
sensitivity is range restriction for each of the stochastic 
parameters. The restriction must be accomplished about the 
value of the parameter used in the deterministic 
reformulation and must provide a reasonable chance for any 
realization of the random variable to be in the restricted 
range. Additional assumptions for this paper are that the 
restricted ranges are symmetric about the parameter value 
used in the deterministic reformulation and that 
perturbations in any given restricted range have no effect 
on perturbations in any other restricted range. 

The individual restricted ranges may be 
developed with statistical confidence intervals, [4] and 
[9], from previous experience, from known physical 
limitations or any other approach which yields a restricted 
range meeting the guidelines established above. The 
specific method used to establish restricted ranges for an 
invariance analysis is not dealt with in this paper. 



The restricted range for a stochastic parameter 

* 

c will he denoted as; 

j 



♦ 

c. = c. + <j>. 

3 3 3 

L L 

-<j> < ^ < 4 . 

3 3 3 

where 

c is used in the deterministic reformulation. 
3 

is a random variable. 

j 

L ♦ 

<{) is the limit for perturbations to c 
j 3 
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b. Changes to the Solution Set 



A requirement of the invariance criterion is an 
ability to identify when a variable must depart the 
complement of the solution set to maintain optimality. For 
an algorithm to be useful for the optimization of the 
deterministic reformulation in the context of the invariance 
criterion it must be possible to perturb a problem parameter 
without necessitating a complete reapplication of the 
algorithm. 



C. SOLOTICN APPROACH 

For arbitrary return functions the stochastic program 
posed by the overhaul planning problem can be optimized 
within the definitional framework of invariance by; 

1. selecting the parameter values to be used in the 
deterministic reformulation, 

2. restricting the range of each stochastic parameter 
about the value selected in step one, 

3. selecting an appropriate algorithm, depending on the 
form of the deterministic reformulation return 
functions, and optimizing the deterministic 
reformulation, 

4. introducing perturbations within the established 
ranges to the stochastic parameters and checking the 
invariance condition. 
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This approach will be illustrated for common classes of 
stage return functions and for each class tests of 
invariance will be identified. The classes of return 
function to be examined are linear, piecewise linear, and 
quadratic. 
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III. LINEAR STA^ RETURN FUN CTIONS 



The simplest function in the general class of concave 
functions is the linear function and while the assumption of 
linear stage returns may not be very realistic in the 
economic context it does provide a useful starting point for 
illustrating the application of the definitional framework 
developed to optimize overhaul planning allocations. 

Assume that each of the n stage return functions is a 
linear function of the form 

♦ ♦ * 

r (d ) = a + c d . 

j j j j j 

The initial steps in the proposed solution approach are to 

* * 

select the values of a and c , respectively a and c , 

j j j j 

for use in the deterministic reformulation and to establish 

restricted ranges about a and c . Given the estimates of 

j D 



a and c with appropriate restricted ranges about them and 

: j 

the general mathematical formulation of the overhaul 
planning problem the linear deterministic reformulation is 
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n 



s/t 





(a . 


+ c d ) 


j=1 




j j 


n 




VI 


BD 


j=i 


j 




VI 

o 


b < 


ub j = 1»2, . . .n 




j 


j 



♦ 

As a affects only the value of the objective function it 

j 

can be dropped from the formulation with no loss of 
generality. The resultant simplified formulation is 



n 

MAX ^ c d 

j=1 j j 



n 

s/t 21 d < BD 

j=1 j 

0 < d < ub j=1,2,...n 

j j 



and the invariance condition need only be checked for the 

♦ 

restricted ranges of c 

j 




* ^ 

L 

< (j) . . 



The deterministic reformulation above is 
program and the most common algorithm for 



a linear 
such an 
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optimization is simplex. The simplex algorithm can accept 
parameter perturbations via the parametric sensitivity 
analysis techniques outlined in Gass [5] and is therefore 
useful for the optimization of the linear deterministic 
reformulation in the context of an invariance analysis. 



As a reference for the investigation of the conditions 
imposed by the invariance criterion, consider the following 
four stage (n=4) example problem. 



MAX 1.4d + 1.6d + 1.3d + 1.7d 

12 3 4 



s/t d + d ♦ d + 
1 2 3 



d < 10 
4 

< 5 

< 3 



< 4 



d < 4 
4 



d > 0 j=1,2,3,4, 

j 



For this problem the optimal simplex tableau is 



B 


c 

B 


1 


2 


3 


4 


5 

% 


6 


7 


8 


9 


d 


1 




1 


0 


1 


0 


1 


0 


-1 


0 


-1 


3 


6 


0 


0 


0 


-1 


0 


-1 


1 


1 


0 


1 


2 


2 


1.6 


0 


1 


0 


0 


0 


0 


1 


0 


0 


3 


8 


0 


0 


0 


1 


0 


0 


0 


0 


1 


0 


4 


4 


1.7 


0 


0 


0 


1 


0 


0 


0 


0 


1 


4 


Z 

j j 


0 


0 


. 1 


0 


1.4 


0 


.2 


0 


. 3 

























where columns five through nine are slack variables. 
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and 



without loss of 



For notational simplicity 
generality the following assumptions are made; 

1. The standard simplex tableaus are used and are 
arranged so that the first n columns are the 

St 

decision variables (d ) . The n+1 column is the 

j 

slack variable for the total budget constraint, 

which is binding at optimality, and columns n+2 
through 2n+1 are the slack variables for each of the 
decision variables. 

2. At optimality there is no degeneracy and no 
alternate optimal solution. 

Notationally let 

E be the optimal solution set or basis. 

C be the vector of the c for j eB. 

B j 

th 

P be the j column of the optimal tableau. 

j 

I be the vector of the 4> for j eB. 

B j 

C be the vector sum of C and I . 

B B B 

It is important to note that for the slack variables both c 

j 

and 4> are zero. 

j 

Linear optimizations have a property that for any 
variable to depart the complement of the solution set a 
variable must depart the solution set. Given this 
one-to-one correspondence between changes in the solution 
set and its complement the requirements of tiie invariance 
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criterion can be satisfied if within the specified 
restricted ranges for the stochastic parameters there are no 
changes to the solution set. In the simplex algorithm 
changes to the solution set can occur only if the algorithm 
optimality criteria are violated. Therefore, the key to an 
invariance analysis in the linear case is the optimality 
criterion of the simplex algorithm. The optimality 
criterion of the simplex algorithm is that 

(z -c ) = c P -c > 0 Vj. 
j j B . j j 



Introducing the perturbations allowed by the restricted 

parameter ranges has the following effect; 

♦ # * 

(z -c ) = c P -c 

3 j B j j 

= (C + I ) P (c J 
B B : 3 3 

= (C P -C ) + (I P -<i> ) 

B j 3 B j j 

or 



♦ 

(Z -c .) = 
3 3 



(z c ) 
3 3 



+ (I 



B 



P - 4>.) 
3 3 



and as the perturbations were introduced at optimality the 
necessary condition to maintain optimality is that 

-(I P - (J> ) < (z - c ) Vj. 

B j j 3 j 



For the solution to be invariant the optimality maintenance 
condition must be satisfied throughout the restricted ranges 
of the stochastic parameters, that is for 



t L 

-(|) < (t> < 4> . 

3 j 3 

The presence of the I term in the optimality maintenance 

B 

condition means that for any column j the maintenance of 
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optimality is dependent on perturbations to parameter c and 

j 

on perturbations to the parameters of the variables in the 
optimal basis. Thus in the optimality maintenance condition 
for any column j there are mutual dependence effects from 
the stochastic parameter perturbations. 

The constraint matrix of the problem has a structure 
which produces an optimal tableau whose elements are either 
minus one, plus one, or zero. Further the assumption that 
the total budget constraint is binding at optimality 
produces an optimal tableau with only four types of columns. 
These column types are identified as follows; 

Type 0 = columns of basic variables. 

Type A = columns of non-basic decision variables. 

Type E = columns of non-basic slack variable. 

Type C = the column of the non-basic slack variable of 
the total budget constraint. 

These special features reduce the mutual dependence effects 
of parameter perturbations in the optimality maintenance 
condition and allow the development of tests of invariance 
for each of the column types identified. 



A. TESTS OF INVARIANCE 



Due to the special structure of the problem the simplex 
algorithm will bring the decision variables into the basis 
at their upper bounds ub sequentially in decreasing order 

j 

of the rates of return. Under these circumstances 
optimality will occur when the entering decision variable 
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causes the total budget constraint to become binding. The 
decision variable which causes the total budget constraint 
to become binding will be the final variable to enter the 
basis and will be denoted d . For the example problem d is 



As shown in Gass [5] parametric perturbations 
produce no change in the optimality conditions for type 0 
columns, that is 



* 

(Z -C ) = (2 -c ) = 0. 

j j j j 

Thus there is no requirement to examine the columns cf the 
optimal basis under the invariance criterion. In the 
example problem columns one, two, four, six, and eight are 
type 0 columns. 

2* Col umn Type A 

Column three of the example problem is a type A 
' column. In general any column j of type A has three 

non-zero elements. These elements are located as follows: 

A plus one in the row of d^. In the example problem 
this is row one. 

A plus one in the row of the slack variable 

th 

associated with this, the j , variable. 
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A minus one in the row of the slack variable 

associated with d . From the tableau assumptions 

f 

the slack variable associated with d is variable 

f 

n+f+1. In the example problem this is variable six 
which is associated with tableau row two. 



As t.he (J) of the slack variables are zero the scalar product 

j 

I P for a type A column must always be 
B j 



I P = d) 

B j f 



and the optimality condition is 



< (z .-c .) . 

f J 3 3 



As (z -c ) is a positive quantity the worst condition for 

j j 

invariance is at 



L 




L 

: 3 



and the test for invariance in a type A column is 



L L 

^ . <" (z .-c .) . 

f : : : 



3* Column Type B 



Examples of type B columns can be found in columns 
seven and nine of the example problem. In general any 
column j of type B has three non-zero elements. These 
elements are located as follows: 
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A minus one in the row of the d . In the example 

f 

problem this is row one. 

A one in the row of the slack variable of d . From 

f 

the tableau assumptions the slack variable 

associated with d is variable n+f+1. In the 

f 

example problem this is variable six which is 
associated with tableau row two. 

A one in the row of the decision variable associated 
with this slack variable, by the tableau assumptions 
this is variable j-(n+1). 

For this type of column the scalar product I P _ is 



B j 



IP = -d + 4> 

B j f j- (n+1) 



and the optimality condition is 




The worst condition for invariance occurs when 



f 



f 



L 






j- (n+1) 



j- (n+1) 



and the test of invariance for a type B column is 
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4. 



Co lumn T ype C 



An example of a type C column is column five of the 
example problem. In general for column (n+1), which is the 
only type C column, there are two non-zero elements. These 
non-zero elements are located as follows; 

A one in the row of d^. in the example problem this 

is row one. 

A minus one in the row of the slack variable 

associated with d . From the tableau assumptions 

f 

the slack variable associated with d is variable 

f 

n+f+1. In the example problem this is variable six 

which is associated with tableau row two. 



The scalar product I P for the type C column is 

B n» 1 

IP =6 

B n+1 f 

and the optimality condition is 

' - <1>^ < (z ,-c .) . 

f 3 3 

The worst condition for invariance occurs when 




and the test for invariance in the type C column is 



L 

p < (z.-c). 
f 3 3 
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The solution set is said to be invariant with respect 
to the specified parameter ranges if the tests of invariance 
are not violated for any of the columns of the optimal 
tableau . 

The tests developed above are sufficient but not 
necessary conditions for invariance. Under certain 
conditions violation of the sufficiency test for a type B 
column may not result in a violation of invariance. This 

+ 

will occur when a variable in S can become zero without 

0 

requiring a variable in S to become positive. The condition 
for this occurence is that it must be possible to allocate 
all of the units of resource from the variable which is 
dropping to zero to the variable which is in the solution 
set but not at its upper bound (d^) . If the violation of the 

sufficiency test for a type B column occurs in column i a 
new condition for invariance should be checked. This 
condition is 



ub > ub - d . 

i-(n+1) f f 

The solution will not be invariant if this condition is 

satisfied since the resource not allocated to d must be 

f 

0 

allocated to some variable in S to maintain optimality. If 

the condition is not satisfied the solution may be invariant 
but to determine whether or not the solution is invariant 
requires additional tableau pivots. 

Recognizing that the simplex algorithm brings the 
decision variables into the solution set at their upper 
bounds sequentially in decreasing order of their rates of 
return and that optimality occurs when a decision variable 
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can enter only at some value less than its upper 
logical statement of a test of invariance is 

♦ ♦ 
max c < min c 

US'" ^ jeS'' 

This condition is equivalent to the tests of 
developed above. 



/ 



bound a 



invariance 
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IV. PIECEWISE LINEAR RETURN FUNCTIONS 



Since piecewise linear functions can exhibit diminishing 
marginal returns they are economically more realistic than 
the' simple linear function. This increase in economic 

realism is gained at the loss of some of the mathematical 
simplicity of the simple linear return model. However, the I 
highly desirable property of linearity is retained. 

Piecewise linear functions generally arise from two sources, 
one is segmented linear regression. The second and more 
common source is the approximation of non-linear functions. 

Assume that the individual stage return functions are 
piecewise linear functions of the form 



« i* I 2« 2 k* k 

a+c d+c d+...+c d 

j j j j j j j 



1 2 k 

d = d + d + ...+d 
j j j D 

m m 

0 < d < ub m=1,2,...,k 

j j 



and to ensure the desired concavity 



m m+ 1 

b > b m=1,2,...,k-^1. 

j j 

Given the economic context of the problem it is reasonable 
to assert that marginal returns are strictly positive. This 
reguires the piecewise linear function to be monotone and is 
ensured if 



t . (<a .) 
3 3 



where 



k 

b > 0. 
3 



2a 



Graphically a typical three segment piecewise linear stage 
return function will have the form 




0 Ub] 



0 ubj 

A further reguirement which ensures that the algorithm used 
for the optimization of the deterministic reformulation 

m + 1 

correctly models the piecewise linear function is that d 

j 

m m 

remain at zero until d is at ub . It should also be 

j j 



noted that 
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k 




ub , Vj, 
3 



thus the original stage constraints remain in force. The 
assumption of k segments for each of the piecewise linear 
functions is for notational simplicity and is made without 
loss of generality. 



m* 

The parameters c can be regarded as approximations 

j 

th 

to the slope of the j stage return function over the 



m-1 m 

interval [ ub , ub ] and as such they are subject to 

j j 



random error. It must be noted that in addition to the 



guidelines for restricted ranges established in chapter two 
the restricted ranges in the piecewise linear case must 
ensure that the function remains concave and monotone 

ml 

increasing. If the limits for a restricted range are -<j) 

j 

mL 

and the maintenance of concavity requires that 

j 

m mL m+ 1 (m+1 ) L 

c - > c + (J> m= 1 , 2 , . . . , k- 1 

j j j j 

and to maintain the monotonic behavior requires that 

k kL 

c - 4 >0. 

j j 



m 

Given the estimates of the c ’s with appropriate 

j 

restricted ranges about them and the general mathematical 
formulation cf the overhaul planning problem the piecewise 
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linear deterministic reformulation is 



n 









MAX 



Z(a . Z 

-»= 1 -i m=1 



m m 
c d ) 

j j 



n k 

s/t ^ ^ d ” < BD 
j=l m=1 j 

m m 

0<d <ub j-1,2, 
3 3 

1 / 2 / 



,n 



,k. 



* 

As a affects only the value of the objective functicn it 

j 

can be dropped from the formulation with no loss of 
generality. The simplified formulation which results is 



MAX 



n k 



Z Z 

m- 1 



m m 
c d 

j j 



n k 

s/t 2! ^ d ” < BD 

j=1 m=1 j 

m m 

0 < d < ub j=1 ,2, . . . ,n 
3 3 

m~1 ^2^. « . ^k 



and the invariance criterion need only be checked for the 

m* 

restricted ranges of c 

j 



m * m m 

c = c + d) 

j j j 
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mL 01 mL 

-4 < 4» < (j> 

j j j 

The deterministic reformulation above is a linear 

m + 1 

program except for the requirement that d remain at zero 

j 

m m 

until d^ is at ub . Fortunately this requirement does not 

prevent the use of the simplex algorithm since, as shown by 

Dantzig [1] and [2], the concavity of the objective function 
ensures that the requirement will be satisfied at 

optimality. Thus the simplex algorithm is applicable for 
the optimization of the piecewise linear deterministic 
reformulation. 

For notational simplicity and without loss of 

generality the following assumptions are made; 

1. The standard simplex tableaus are used and are 
arranged so that the first nk columns are the 

m st 

decision variables (d ). The nk+ 1 column is the 

j 

slack variable for the total budget constraint, 

which is binding at optimality, and columns nk+2 
through 2nk+1 are the slack variables for each of 
the decision variables. 

2. The column arrangement is by block according to the 
variable index j and within each block the k 
segments are in order from one to k. Thus the column 
arrangement is of the form 

12 k 1 k 1 k 

d d ...d d ...d d d ...d . 

11 12 n n+1 n+2 2n+1 



32 



3. There is a tableau row for each variable to enforce 
its respective bound. Thus the tableau will have 
nk+1 rows and 2nk+l columns. 



4. At optimality there is no degeneracy and no 
alternate optimal solution. 



Notationally let 

B be the optimal solution set or basis. 



m m 

C be the vector of the b for d eB. 
B j j 



m 

P be a column of the optimal tableau. 

j 



m m 

I be the vector of <i> for d e B. 
B j j 



C be the vector sum of C and I . 
B B . B 



The optimization of piecewise linear functions is a 
linear optimization and as such there is a one-to-one 
correspondence between changes to the basic and non-basic 
variables. However in the piecewise linear case this is not 
equivalent to a one-to-one correspondence between changes to 
the solution set and its complement. This is due to the fact 
that in the piecewise linear case invariance is directed at 
the original variables d not at the piecewise segment 

j 

m 

variables d- . The invariance condition can be stated as; 

j 

if within the specified restricted ranges for the stochastic 



1 0 

parameters the d which are in S remain at zero the 

j 
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solution is invariant. This condition can be checked through 

the simplex algorithm optimality criteria. In the notation 
of the piecewise linear case the optimality criterion of the 
simplex algorithm is 



mm m m 

(2 -c ) = C P -c >0 Vj# m=1,2/...,k. 
j j 3 j j 



Introducing the perturbations allowed by the restricted 
parameter ranges has the following effect; 



m m * * m m * 

( 2 -C ) = C P - C 

3 3 3 j j 



or 



m mm 

= (C +I)P. -(c. + <).) 

B B g 3 3 

mm mm 

= (C P . -c . ) + (I P . - p , ) 

3 : 3 B 3 3 



mm* mm mm 

<Z. - c . ) =(z. ~C,) + (IP. - 

ID D D B 3 3 



and to maintain optimality 



in in mm 

-(IP -(j>)<(2 -C). 

B j j j j 



The invariance criterion requires that the optimality 
maintenance condition be satisfied throughout the specified 
restricted ranges of the parameters, that is for 



mL m mL 

-<J> < < (j> 

j j D 



The piecewise linear optimality maintenance condition 
is, with notational differences, the same as the simple 
linear case. The only real differences are that there are 
now nk total decision variables instead of the n for the 
simple linear case and that the optimality maintenance 
condition need not be checked for all of the columns cf the 
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optimal tableau for the solution to be invariant. The 
columns for which the optimality maintenance condition need 
not be checked are those of the non-basic structural 
variables which do not affect invariance, that is the 
m 

columns of d where m>1. These variables do not affect 

j 

invariance since the invariance condition for the piecewise 

1 

linear problem is stated in terms of the d which are in 

j 

o 

S . The special features of the problem, namely the 

constraint matrix structure and the assumption of a binding 
total budget constraint, are intact and the optimal tableau 
has the same four column types and element arrangements as 
the simple linear tableau. Thus the invariance analysis for 
piecewise linear functions is virturally identical to that 
of the simple linear case. The difference is that the only 
type A columns which must be checked for invariance in the 

1 

piecewise linear case are those associated with d 's which 

j 

0 

are in S . Since the analysis for the columns which must be 

checked is the same in the piecewise linear problem as in 
the simple linear problem the tests of the simple linear 
case will simply be restated in the current notaxicn for 
those columns of the piecewise linear case which are 
relevant under the invariance criterion. 



A. TESTS OF INVARIANCE 



As in the simple linear case the subscript f denotes the 
decision variable which upon entry into the basis causes the 
total budget constraint to become binding. 



1 
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1 



♦ Col umn Ti£e J (basi c variabl es) 

No examination of the columns of the optimal basis 
is required under the invariance criterion. 

2 • Column T22® A (uon-basic §^uc tur al variab les) 

As pointed out above the only columns of type A 
which are relevant under the invariance criterion are those 

1 

associated with d 's. The test of invariance for a relevant 

j 

type A column is 



2* Colu^ Ti£e 3 (non- basic slack v ariable s) 



The test for invariance- for any column of type B is 




< (2 - c ) . 

j j 



1 1 



L 



m m 



f 



+ 4. 




The test for invariance for the type C column is 



L 



f 



< (2 



nk-t^l 



- c 



nk + 1 



) . 
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The solution set is said to be invariant with 
respect to the specified parameter ranges if the tests of 
invariance are satisfied for all relevant columns of the 
optimal tableau. 

As in the simple linear case the tests above are 
sufficient but not necessary conditions for invariance. 
Dnder certain conditions a violation of the type B column 
sufficiency test may not be a violation of invariance. This 

+ 

occurs when a variable in S can become zero without 

0 

requiring a variable in S to become positive. The condition 
for this occurence is that it must be possible to allocate 
all of the units of resource from the variable which is 
dropping to zero to the variable which is in the solution 
set but net at its upper bound (d^) . However in the 

piecewise linear case it is not possible to state a 

condition which checks for this behavior. This is due to the 

fact that violation of the allocation condition stated above 

may not result in a violation of invariance since the units 

of resource not allocated to d may be allocated either to a 

f 

m 

d where m> 1 , which would not violate invariance, or to a 

j 

1 

d which would violate, invariance. 

j 

As in the simple linear case the simplex algorithm 
will bring the decision variables of the piecewise linear 
case into the solution set at their upper bounds 
sequentially in decreasing order of their rates of return. 
Optimality will still be achieved when an entering decision 
variable encounters the total budget constraint and the 
logical statement of the tests of invariance remains 
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m* 

max c < min c m=1,2,...k. 

j j65* : 

The solution is said to be invariant if this condition is 
satisfied throughout the specified restricted ranges. 
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V. Qa ADRA TIC ST^E RETU RN FUNCTIONS 



Quadratic functions are another move toward economic 
realism for the overhaul planning problem stage return 
functions. The advantage of a quadratic function is that 
marginal returns can be made to be continuously decreasing 
whereas the piecewise linear function could only 
incrementally decrease marginal return. The gain in realism 
with the quadratic function has a large price, the loss of 
linearity. However this loss is partially offset since rhe 
quadratic function is one of the simplest of the non-linear 
functions and there is some advantage over piecewise linear 
functions because . of the reduced number of parameters and 
variables. 



Assume that each of the n stage return functions is a 
quadratic function of the form 



* ♦ 1*2 

r (d ) = a d - b d . 

3 i 3 3 2 j j 



The conditions for the desired properties of decreasing but 
always positive marginial returns, or monotonic concavity, 
are that the first derivative be non-negative and the second 
derivative be strictly negative. This means that 

a > b ub 

j j j 

b >0. 

3 



It should be noted that 
quadratic case must, 
guidelines of chapter two. 



the restricted ranges in the 
in addition to satisfying the 
maintain the monotonic concavity 
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of the individual stage return functions. If the limits of 

♦ 3}C X# 

the restricted ranges of a and b are respectively ±<|> 

j j j 

L 

and ±0 the maintenance of monotonic concavity requires 

j 

that 



b - 9 >0 

j j 

and 

L L 

a - <b > (b +0 ) ub 

j j j j j 



Given the estimates of a and b with appropriate 

j j 

restricted ranges about them and the general mathematical 

formulation of the overhaul planning problem the quadratic 
deterministic reformulation is 



1 2 

MAX a d -> b d 

j=1 j j 2 j j 



s/t d < BD 

j = -5 j 



0 < d < ub j=1,2,...n. 
3 3 



The invariance condition must be checked for the restricted 

♦ 



ranges of b 



b = b + 9 

j j j 
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L L 

-9 < 0 < 0 

3 3 3 



* 

and for the restricted ranges of a 

j 



♦ 

a = a + (i 

j j j 



3 3 3 



It will be notationally convenient to shift to matrix 
notation where 



D is an (nxl) vector of decision variables d . 

j 



E is an (nxl) vector of parameters b . 

j 



A is an (nxn) positive definite diagonal matrix of 

parameters a . 

3 



I is an (nxn) identity matrix. 



♦ 

1 is an (nxl) vector of 1*s. 

UB is an (nxl) vector of the stage constraints ub . 

j 



The deterministic reformulation can be written in matrix 
notation as 



t 1 t 

MAX H(D) = A D - — B DB 



t * 

S/t D 1 < BD 



0 < ID < UB. 



This is a convex non-linear program and the Kuhn-Tucker 
conditions ate necessary and sufficient for optimality. 
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The initial attempt to optimize the deterministic 
reformulation and apply the invariance criterion involved 
the use of quadratic programming, specifically the algorithm 
of Dantzig as described by Boot [3]. While this algorithm is 
simplex based and technically satisfies the requirements for 
the application of the invariance criterion it involves a 
reformulation via the Kuhn-Tucker conditions and the 

resultant problem proved to be conceptually and 

computationally difficult. The specific difficulties were; 

1. The reformulated problem is conceptually an 
existence problem rather than an optimization 
problem. Thus there is no objective function in the 
usual sense and this prevents the determination cf 
optimality maintenance conditions. 

2. The parameters to which perturbations are applied 
are in the constraint matrix and on the right hand 
side of the reformulated problem. This situation is 
almost impossible to handle when perturbations are 
applied to more than one parameter. 



The second algorithm selected for the optimization was 
the reduced gradient algorithm, [6], [7], and [10]. This 
algorithm is very similiar to the simplex procedure, 
identifing variables as either basic or non-basic. One of 
the important differences between this algorithm and simplex 
is that here the basic variables must be positive and the 
non-basic variables may be different from zero'. The reduced 
gradient algorithm can also handle parameter perturbations 
without having to reapply the entire algorithm, thus the 
algorithm is applicable under the requirements of the 
invariance criterion. To put the deterministic equivalent 
into the standard format of the reduced gradient algorithm 
add slack variables to the constraint matrix and denote the 
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result as 



t * ' ' 






D 1 1 1 '0* 

( 




BD 


1 1 
! \ ^ 

1 * ‘ 


< 




ID 1 0 1 1 




UB 

_ 



where 

0* is an (1xn) vector of zeros. 
♦ 

0 is an (nxl) vector of zeros. 



additionally at optimality let 

Z be the set of the n+1 basic variables. 

N be the set of n non-basic variables. 

P be the matrix of the columns of W associated with 
the basic variables. 

C be the matrix of the columns of W associated with 
the non-basic variables. 

th th -1 

g be the 1 element in the k row of Q = P C. 

It should be noted that 

+ 

1. The solution set S has elements from both Z and N. 

0 

2. The complement of the solution set S has elements 
only frcm N. 

3. The column arrangement of W is the same as the 
column arrangement of the constraint matrix in the 
linear case. 

4. For the slack variables ^ 6 , and d are all zero. 

3 3 3 
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In the quadratic case there is no one-to-one 
correspondence between changes in the solution set and its 
complement. Thus the use of the invariance criterion in the 
quadratic case requires a direct application of the 
definition cf invariance, that is the invariance analysis 

0 

must be focused on the complement of the solution set, S . 

The maintenance of optimality in the quadratic case is not 
necessary for invariance, in fact optimality will probably 
never be maintained. However, the optimality condition of 
the reduced gradient algorithm does provide a method for 
checking the invariance criterion. The optimality condition 

of the reduced gradient algorithm is that all elements u of 

D 

the movement vector U, defined below, be equal to zero. The 
elements of the movement vector are defined as 



u 

j 



< 



if RG < 0 and d 

j j 



0 



V jeN 



HG otherwise 

j 



th 

where RG is the j ’ component of the reduced gradient RG 

j ' 

and 



RG 



-? R(D) 
Z 



- 1 

P c 



7 R (D) 
N 



or 



RG 



j 



^ (a - bd)q +(a - bd). 

zeZ z zzz,j j jj 
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The key for the invariance criterion is that the 

0 

variables in the complement of the solution set (S ) are 

specifically required to have a reduced gradient component 

0 

which is non-- positive. Departures from S can only occur if 

the perturbations within the specified restricted ranges 
cause the reduced gradient component of one of the variables 
0 

in S to become positive, thus invariance is dependent on 

the changes to the reduced gradient components of the 
0 

variables in S produced by the parametric perturbations. 

Introducing the parameter perturbations allowed by the 
restriced ranges has the following effect on the reduced 
gradient . 



RG = - ^ (a -bd)g +a -bd 
j zeZ z 2 zz,j j jj 

= - (a +<{) ”(b + 9)d)q 

zcZ z z z z z z,j 

+ (a + (j))-(b +9)d 

j j j j j 

= - (a - bd)q -►a - bd 

zsZ z z z z,j j j j 

- 21 {<^ - 0d)q + p - 9d 

zeZ z z z z,j j j j 



= RG -A(<> - 0d)g +(^ - 9d. 

j z^ z zzz,j j jj 



* 0 

For invariance the RG of the variables in S must remain 

j 

0 

non-positive. Noting that d for all variables in S must be 

j 
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zero the changes in the reduced gradient components of the 
0 

variables in S simplifies to 



EG =RG-^(4)-0d)q +4. 

j j zeZ z z z z,j j 

Since EG is a non-positive quantity the condition for the 

j 

♦ 

maintenance cf non-positive EG can be expressed as 

j 

V. ” 0 

EG >-Z_(4> - ed)q +<)) V jeS 

j zeZ z zzz,j j 

and for invariance this condition must be maintained for all 

0 

variables in S and throughout the specified restricted 
ranges, that is for 



L 

: : 



< 






L 



L L 

-9 < 0 < 0 . 

j j D 



The condition for the maintenance of non-positive EG 

j 

0 

for the variables in S can be simplified by examining the 



structure of the constraint matrix W, which has exactly the 



same structure as the constraint matrix of the linear case. 
Although in- the quadratic case there can be more than one 
variable in the solution that is not at its upper bound the 
structure of H allows for the occurence of only one of them 
in the basis set Z. This arises as follows; 



The occurence of a variable which is not at its upper 
bound in the basis set Z requires that both the variable 
and its slack be in Z. This requires that the 
(n+ 1) X (n* 1) matrix P, which must be non-singular. 
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contain both the column of the structural variable and 
the column of the slack. Each variable/slack pair after 
the first fills two column dimensions but only one row 
dimension in P. If an attempt is made to identify i>1 
variables as not being at their upper bounds there will 
be (n- i+1) filled column dimensions but only (n-i) 
filled row dimensions in P. Of the remaining (n - i + 
1) variables in the basis, each is identified as being 
at its upper bound and fills only one row and only one 
column dimension in P. Thus the net result of trying to 
identify mere than one variable as not being at its 
upper bound is an unfilled row dimension in P and this 
row of zeros makes P singular, which is in violation of 
the requirement for P. 



Since only one variable can be identified as being in the 
solution but not at its upper bound the basis in the 
quadratic case is exactly the same as in the linear case and 
the representation of the non-basic columns of W, that is Q, 
at optimality is exactly the same as in the linear case. 
This means that the columns of Q associated with the 
0 

variables in S have the same structure as the columns of 



the linear case which are associated with non-basic, and 

therefore zero valued, variables. The columns of Q 

0 

associated with variables in S are either type A, B, or C 
columns where 



type A = columns associated with structural variables. 

type B = columns associated with slack variables. 

type C = the column associated with the slack of the 
total budget constraint. 
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A 



TESTS OF INVARIANCE 



Denote the set of variables which are in the solution 
but not at their upper bounds as the set F. The variable of 
F which is identified as being in the basis but not at its 
upper bound will be denoted as d^. With this notation the 

arrangement of the elements in the quadratic type A, B, and 

C columns is identical to the arrangement of the elements in 
the linear type A, B, and C columns. For details of these 
arrangements see pages 22, 23, and 25 respectively. 



1 . Colu mn Typ e A (stru ct ura l vari ables) 



0 

For d e S that are structural variables the 

j 

columns of Q will have the structure of the type A column. 



♦ 

see page 22. Under this condition the summation in the RG 

j 

maintenance condition becomes 



-( 




- 0 d ) 
f f 



and the condition for the maintenance of non-positive 
is 






RG 



i 



RG 



-( - 



9 d ) 
f f 



i> 
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a 



To obtain a test of invariance the variable identifed as 
being in the solution but not at its upper bound must be the 
variable in J which has the largest deviation from its 
optimal value marginal return, that is select d^ to 

L L 

max ( d> +9 d ) . 

Uf f f f 



The test of invariance for a type A column is 

L L . L 



I ) ij L. U 

RG > max ( <^ + 9 d ) + <j> . 

1 jl f f f j 



2 . Cc^ran Ti£e B (slac k v ariabl es) 



For d € S that are slack variables the columns of 

j 

Q will have the structure of the type B column, see page 23. 



Under this condition the summation term in the RG 



maintenance condition becomes 



-(<b-9d)-(<k -9 d ). 

"^f f f j-(n+1) j-(n + l) j-(n+1) 

The test of invariance for type B columns is 



II L L 

RG I > max ( (> ) 

' T fsF f f f 



L L L 

+ (<b — 9 d 

j-(n+1) j-(n+1) j-(n+1) j 
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4 





3. 



IXEJ C ibiid£et constraint sla ck v ariabl e) 

The slack variable associated with the total budget 

constraint is variable n+1. The column of Q associated with 

d is a type C column, see page 25. Under this condition 
n+1 

the summation term in the RG maintenance condition becomes 

j 

- ( <|> -9 d ) 

f f f 

and the test of invariance is 



II L L 

RG > max ( + 9 d ) 

' feF ^ f 



3 ' 

The solution is said to be invariant with respect to the 
specified parameter ranges if the tests of invariance hold 
0 

for all d € S . 

3 



The tests developed above are sufficient but not 
necessary conditions for invariance. Under certain 
conditions violation of the sufficiency test for a type B 
column may not result in a violation of invariance. This 

+ 

will occur when a variable in S can become zero without 

0 

requiring a variable in S to become positive. The condition 
for this cccurence is that it must be possible to allocate 
all of the units of resource from the variable dropping to 
zero to the variables in the solution set that are not at 
their upper bounds. If the sufficiency test for a type B 
column is violated for slack variable i a new condition for 
invariance should be checked. This condition is 
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d > ^ (ub - d ) . 

i- (n+1) fsF f f 

If this condition is satisfied the solution is not invariant 
since the units of resource not allocated to the variables 

0 

in F must be allocated to a variable in S to maintain 

optimality. If this condition is violated the solution may 
be invariant and to determine whether or not the solution is 
invariant requires additional algorithm iterations. 

As in the linear case there is a logical statement 
of the tests of invariance. This is derived by recognizing 
that the reduced gradient algorithm will allocate 

incremental units of resource in decreasing order of the 
maximum marginal returns. At optimality all variables in the 
solution but not at their upper bounds will have a common 
marginal rate of return A, As long as the parametric 
perturbations allowed by the restricted ranges do not cause 

0 

the marginal return at zero of a variable in S to become 

larger than the marginal return of any variable in S the 

solution will be invariant. This test of invariance is 
equivalent to the tests developed above and can be expressed 
as 

L L L 

max (a + <h ) < min (a + (J> )-(b + 9 )d 

jeS" j j 33 j D j 

If within the specified restricted ranges the condition is 
satisfied the solution is said to be invariant. 
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VI. CONCLO SIGNS AND EX TENSIONS 



The invariance criterion and the definitional frameworJc 
in which it is applied provide an approach toward the 
optimization of the stochastic program posed by the overhaul 
planning problem. The complement of the optimal solution set 
of the deterministic reformulation identifies the stages in 
which no advance planning and procurement should be 
undertaken and the sqlution sensitivity analysis under the 
invariance criterion provides some control over the risk 
involved in basing decisions on a deterministic solution to 
a stochastic problem. The key is that the risk is only 
controlled since even if a solution satisfies the invariance 
criterion there is a positive probability that the 
realizations of the stochastic parameters will fall outside 
the restricted ranges used for the invariance criterion. 
Thus even if a solution satisfies the invariance criterion 
it should not be regarded as an exact answer, but as a 
foundation on which objective management decisions can be 
based. Even if a solution fails the tests of invariance 
there is a positive result, the stages which are susceptible 
to stochastic behavior can be identified. For a solution 
which fails to satisfy the invariance criterion there are 
two ways to move toward invariance. 

1. Reduce the variance of the estimate used in the 

deterministic reformulation. This generally 

reguires an enlargement of the data base used in the 
estimation. 

2. Reduce the restricted ranges on which the tests of 
invariance were based. This will increase the risk 
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associated with decisions based on the solution as 
there will be a greater probability of realizations 
of the stochastic parameters falling outside the 
restricted range. 



A significant improvement in the invariance criterion 
could be achieved if the level of risk associated with an 
invariant solution could be specified. This would require 
the development of a probability statement addressing the 
joint probability of violating the specified restricted 
ranges of the stochastic parameters. Unfortunately a 
realistic development of this probability statement would 
require dropping the assumption of independence for the 
parametric perturbations and would necessitate the 
development cf joint restricted ranges for the dependent 
parameters. This will greately complicate the mathematics of 
the invariance analysis. 
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